Computing the Kullback-Leibler Divergence between two Weibull Distributions

We derive a closed form solution for the Kullback-Leibler divergence between two Weibull distributions. These notes are meant as reference material and intended to provide a guided tour towards a result that is often mentioned but seldom made explicit in the literature. 1 The Weibull Distribution TheWeibull distribution is the type III extreme value distribution; its probability density function is defined for x ∈ [0,∞) and given by f(x | k, l) = k l x l )k−1 exp [ − x l )k ] (1) where k > 0 and l > 0 are shape and scale parameters, respectively. This is a rather flexible, unimodal density. Depending on the choice of k and l, it may be skewed to the left or to the right. For k = 1, the Weibull coincides with the Exponential distribution and for k ≈ 3.5, it approaches the Normal distribution. A excellent account of the origins of the Weibull distribution is given in [1]. Among others, it was introduced as a plausible failure rate model [2] and has been frequently used for life-time analysis in materialor actuary studies ever since. Extending its classical applications, it was reported to account well for statistics of dwell times onWeb sites [3], times people spend playing online games [4], or the dynamics of collective attention on the Web [5]. The Weibull also attracts interest in machine learning or pattern recognition where it was found to represent distributions of distances among feature vectors [6], has been used in texture analysis [7,8], or was shown to provide a continuous characterization of shortest paths distributions in random networks [9]. Accordingly, methods for measuring (dis)similarities of Weibull distributions are of practical interest in data science for they facilitate model selection and statistical inference. 2 The Kullback-Leibler Divergence The Kullback-Leibler (KL) divergence provides a non-symmetric measure of the similarity of two probability distributions P andQ [10]. In case both distributions are continuous, it is defined as